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ABSTRACT . -, , 4. * 

During 1970 and 1971, the National Assessment of 

Educational Progress (NAEP) conducted its first assessment of^ 

reading, measuring the achievement of specific reading objectives by 

individuals aged 9, 13, 17, and 26-35. In 1974, the Right to Read 

T-ffort directed that a Mini-Assessment of Functional literacy (MAFL) 

be conducted to determine be sic functional reading abilities of 

selected 17 year olds still in school. This document is a compilation 

of the results of the first NASP reading assessment and the 

change-in-achievement results for the HAFL in-school assessment of 17 

year olds. Background information and results for both tests are 

d-!scussed, with tables of findings illustrating the text. (JM) 
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Highlights and Trends from NationcU Assessment: 
Changes in Reading Achievement^. 19^70-1975 

by 



Susan J» Oldefendt 



National Assessment of Educational Progress 



National Assessment of Educational Progress (NAEP) con- 
ducted its first assessment of reading during 1970rl971» This 
assessment was designed and conducted in accordeuice with the 
model used ixr other subject area sissessments* First # the 
reading objectives were formulated and reviewed by panels of 
scholar^r educators r students and lay citizens* Then exercises 
(items) ware developed to measisre achievement of these objectives 
by individuals of ages 9, li, 17 and 26-35 • Data which were 
collected from national probability samples have been analyzed 
and summarized ia JUREP reports.. The groups for which achieve- 
ment results were reported eure listed below* 
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In 1974 the National Right to Read Effort awarded a grant 
to the Education Commission of the States to have NAEP conduct 
a Mini-Assessment of Fvmctional Literacy (MAFL) . The effort 
was designed to gather data to determine how well 17-year--olds 
who are still in school can perform basic functional reading 
tasks. NAEP exercises which were judged by reading specialists 
appointed by the Right to Read Effort to be examples of basic 
everyday reading tasks were selected and administered to a 
national probability sample of 5,200 17-year-olds. Many of the 
exercises had been used in the 1971 assessment, and therefore 
it has been possible to compute and report changes m achieve- 
ment on the fvmctional literacy exercises for 17-year-olds 
in-school. 

During the school year 1974-1975, the second NAEP reading 
assessment collected data from 9-year-olds, 13-year-olds and 
17-year-olds, both in-school and out-of -school . Exercises used 
irt 1974-1975 had also been administered in the first reading 
assessment, and the changes in achievement between the two 
times are ciirrently being analyzed. The first NAEP report of 
the change findings is scheduled for early Fall, 1976. A 
replication of the first MAFL assessment was conducted in con- 
junction with the regular 1974-1975 assessment and the MAFL 
Right to Read report is scheduled for release in September, 1976. 

. This paper is a compilation of results from th'- Tirst NAEP 
reading assessment and change results for the MAFL xn-school 
17 -year-old assessment. Changes in achievement levels of 
reporting groups on the functional literacy set of exercises 
are highlighted. 



I. THE FIRST NATIONAL ASSESSMENT OF READING 1970-1971 

Development of Objectives, Exercises and Reporting Cl assifications 

The initial step in the creation of the first reading assess- 
ment was the development of six major objectives {with subobjec- 
tives) for reading. The first five objectives represented the 
individual's ability to comprehend, analyze, use, reason logically 
from and make judgments concerning what he has read. These 
objectives form a hierarchy of reading behaviors; that is, one 
must be able to comprehend a passage before he can analyze it 
and analyze a passage before he cein use it, reason from it or 
make judgments concerning it. This hierarchical arrangement of 
behaviors represents a logical progression of what a student 
should be able to do as a result of his reading experiences and 
instruction. The sixth reading objective was concerned with 
attitudes toward and interests in reading. 



^For~a complete statement of the Reading objectives and a detailed 
discussion of their development, refer to Reading Objectives , 
1970-71 National Assessment of Reading (Denver, CO: National 
Assessment of Educational Progress, 1970). 
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Once the reading objectives were formulated, they became 
the framework within which, the reading exercises were developed. 
Each exercise f therefore, purported to measure readers' skills 
related to one of the above objectives. 

The majority of the reading exercises were in a multiple- 
choice format that included an "I don't know" response. The 
respondent merely filled in an oval beside the response he 
believed to be correct. Some exercises were open-ended, 
requiring the respondent to supply his own response. 

After the reading exercises were developed, another frame- 
work for classification was constructed. The themes were devel- 
oped by members of the National Assessment staff and reading 
specialis.ts to cluster the reading exercises into reporting 
categories that are meaningful to lay persons, scholars and 
educators concerned with the reading skills of various groups 
of yovmg Americans. 

A theme defines a set of existing or potential exercises 
that relate to each other in content or in some central idea 
that is meaningful to the subject area of concern. The reading 
themes represent both a variety of reading materials and a 
variety of behaviors these materials require. The themes are 
numbered--the lower numbers designating the more basic reading 
skills and the higher numbers designating higher-order skills. 
Successful performance on exercises in the higher-numbered 
themes requires some of the abilities measured in the lower- 
ntambered themes in addition to the higher-ordered skill. 

Theme 1: Understanding Words and Word Relationships 

Theme 2: Graphic Materials 

Subtheme A: Interpret drawings and pictures 
Subtheme B: Read signs and labels 
Subtheme C: Read charts, maps and graphs 
Subtheme D: Read forms (such as applications, 
report cards, etc.) 

Theme 3; Written Directions 

Subtheme A: Understand written directions 
Subtheme B: Carry out written directions 

Theme 4: Reference Materials 

Subtheme A: Know appropriate reference sources 
Subtheme B: Use reference materials effectively 

Theme 5: Gleaning Significant Facts from Passages 

Theme 6: Main Ideas and Organization 

Theme 7: Dravjing Inferences 

Theme 8: Critical Reading 
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Summary of Analysis and Results 

For each exercise at each age the percentage of respondents 
in each reporting group who got the exercise correct was calcu- 
lated. Jack-knifed standard errors for the percentages were also 
computed. Differences in percentages for each reporting group 
and the nation were calculated in order to determine whether the 
group Wets performing above or below the nationaa level of per- 
formance. The exarcise-by-exercise results indicate relative 
achievement: levels of groups on specific pieces of content. 

In order to summarize results for each reporting group for 
sets of exercises r medians of, differences in percentages between 
the group and the nation - were calculated. Medians were computed, 
for the entire set of exercises and for certain subsets of 
exercises which were classified in the same objective or report- 
ing theme. By comparing median differences for a group, one 
can determine whether the group always performs above or below 
the nation no matter what the level of the objective or theme 
is, or whether on certain types of reading tasks the group devi- 
ates from its usual pattern of performance. 

Results for each of the reporting groups on the total set 
of reading exercises are presented in Exhibit 2. 

Exhibit 2. Median Differences Between the Nation's 
Performance ^ wfl the Performances of Each 
Group- for the Set of All Exercises 
administered in 1970-71 




♦Number of Differences Summarized 

2summary results for each, reporting them e cart he found in. 
Rfltaor t Q2-R-00r Readino^s- Summary Data (Washington,. D.Cs 
Government Printing o££ice^ L974f^ Detailed reports on each 
of the eight themes^ plus one on reading, rate andi compre- 
hension r are ctlso available • 
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The contrast most evident in the sxommary data was the 
difference between the performance levels of blacks and whites. 
The difference was extremely large, with blacks being far below 
the national level on every theme, without exception. The over- 
all level of performance for blacks, in fact, was lower than 
that for any other group in 1970-1971. That held true at all 
age levels, with black adults having the lowest relative overall 
performance. Among the school-age population, however, the gap 
between blacks and whites did not appear to increase with age. 
That is, in relation to the nation, blacks did not seem to read 
less well at age 17 than they did at 13 or 9 years of age. 

Obvious differences in reading performance also existed 
within all other categories of respondents examined in the 
reading assessment. The overall reading performance of school- 
age boys was clearly below that of girls. Exceptions to that 
pattern were as follows. Nine-year-old boys read about as well 
as girls in the skills areas of vocabulary and graphic materials. 
Males in general also had better recall of specific details than 
girls did on reading rate exercises, though girls generally read 
faster. Among the young adult population, men and women showed 
cibout the same reading ability. 

There was also a relationship between reading achievement 
and the region of the country in which a person resides. 
Specifically, respondents at all four age levels m the south- 
east performed below their covmterparts from the other three 
regions of the country in every reading skill area assessed-- 
from following simple directions to reading critically. South- 
east young adults, moreover, fared even less well than any o£ 
the school-age groups in the southeast. Those adults had 
particular difficulties. in understanding words and word relation- 
ships amd in dravjfilig- Inferences. 

Western school-age respondents tended to be at or slightly 
below the national level; however, western adults were somewhat 
above the national level on every theme and objective. In^ con- 
trast, adults from the northeast performed at about the national 
level on five of the eight themes, whereas northeastern 9, 13 
and 17-year-olds were somewhat better than the national level on 
most themes. Central adults did not do relatively as well as 
the school-age populations from that region. The rentral adults 
were at the national level on three themes— main ideas and 
organization, drawing inferences and critical reading— and some- 
what above on the others. However, all three school-age groups 
from the central region were above the national percentage on 
all themes. The 17 -year-olds from that region performed espe- 
cially well on graphic materials, written directions and reference 
materials. 

The level of parental education also made a difference in 
reading performance ► School-age populations and young adults 
whose parents have not completed high school read far less well 
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than did people of the same age level whose parents havev more 
education. When at least one parent in a family had graduated 
from high school r the chances increased that the school-age 
children Or 13 and 17 -year-olds) would be at least average 
readers compared to the nation as a whole. Coming from a home 
with little formal education seemed to be less of a handicap 
to reading performance for young adults. The highest overall 
reading performance of any parental education category wais 
registered by respondents whose parents had some education 
beyond high school. This was true at each of the fo\ir ages. 

There was also dsi obvious relationship between reading 
proficiency on the National Assessment reading exercises and 
the size and type of community (STOC) which one's schocpl 
serves. School -age young people from the low metropolitan 
group performed far below those in other STOC groups r with the 
greatest difficulty being experienced by the 9-year-olds in 
that STOC category. In contrast r 9, 13 and 17-year-olds from 
the high metropolitan group were superior to all other STOC 
groups in their overall reading performance. 

Between those two extreme groups there was a wide range 
of performance for the other STOC groups in relation to the 
nation as a whole. Because these STOC categories are partially 
related to family income, reading success is apparently asso- 
ciated with parents' financial well-being. 

For young adults the pattern of reading success in relation 
to the nation as a whole was somewhat connnensurate with the size 
of community r as follows: adults from big cities read somewhat 
below the national average, adults from small places and medium 
cities read at about the national average and adults from urban 
fringes read somewhat above the national average. 
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II. THE 1974 MINI-ASSESSMENT OP FUNCTIONAL LITERACY 
Baclcqrottnd InformatdLoa 

In an effort to gather data regarding baisic reading 
abilities of young Americans who were about to finish thexr 
high school educations , the National Right to Read Effort 
and NAEP conducted a Mini-lVsaessment of Functional Literacy 
(MAFL) in 1974. Thia assessment was admini stered to 5,200 
17 -year-olds enrolled in school. They were selected to be 
representative of the entire population of in-school 17-year- 
olds in America. The young people were asked to answer 
questions and complete tasks in reading which had been 
selected by a panel of reading specialists appointed by the 
Right to Read. These exercises were judged to be examples 
of basic everyday reading tausks. According to the special- 
ists, all 17-year-olds should be able to correctly complete 
the MAFlTexercises if they are to function adequately in 
everyday life. 

The exe2.3ises included such questions as: 

+ Here are pictures of four doors you might find in a 
school. Fill in the oval under the door where you 
might go for lunch. (The doors were labeled Nurse, 
Principal, Cafeteria and Library.) 

+ How many additionetl books must you buy? (One of 
several questions about a reprintecl book club 
announcement • ) 

+ Must a person appear at the Traffic Violation 

Bxireau to plead *not guilty"* to a traffic offense? 
cone of several questions eibout a replica of a 
trcLffic ticket.) 

There were 86 such exercises, designed to find out about 
people's basic reading skills. Sixty-four of these exercises 
were duplicates from the 1971 reading assessment. It is on 
these 64 exercises that comparisons between the 1971 and 1974 
assessments are raade._ 

While society might wish that all 17-year-olds attending 
school could perfomt each of the very basic reading tasks in 
the mini-assessment, NAEF and Right to Read staff members felt 
that might not be a realistic standard. A more realistic 
level of achiev«nentr they decided, might be found by asking 
a group of 17-year-old "superior readers* to da the tasks 
and finding out what percentage could da so correctly. 

A superior" reader was def dined, ets a 17-xear-old student 
who bad attained, at least the 90th percentile on the College 
Entrance Examinatioa Board reading test or art equivalent 
standardized reading test. One hundred such, young people 
were located in a metropolitan airea. and con^leted the M2^ 
exercises. The percentage of superior readers who responded 



correctly on each exercise was considered to be the highest 
expected level of performance'' for that exercise. The per- 
centage of all ITryear-olds in the national sample who 
responded correctly on each exercise was adjusted by convert- 
ing the figure to a percentage of the highest expected level 
of performance for that exercise^ rather than measuring it 
against the ideal of 100 percent* All summary values (mean 
P-vaLLues) presented here are beised upon data adjusted to 
relate to the highest expected level of performance. 

Change Results for 1971-1974s - Sevente en-Year-Olds 

The summary statistics which have been compared to deter- 
mine if any change in performance levels oh the functional 
literacy exercises occurred are mean percentages of the nation 
and each reporting group who gave correct ' responses on a set 
of exercises* Mean percentages correct (mean P-values) were 
computed for each reporting group in each year. A summary 
table of the means for the two years and the changes which 
occurred is presented in Exhibit 3. 

Exhibit 1. Mean Percentages of Each Group 
That Gave Correct Responses 
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All groups showed a gadLn in basic reading performance as 
measured in the mini-assessment* While this does not mean 
that all groups gained on all the exercises, all groups did 
show a gain in their tman percentage correct. The average 
percentage of the national sample of 17-year-old students who 
could perf om the basic reading tasks was 2 percentage points 
higher in 1974 than in 1971 • 

In generally those groups gained most who had most to gain* 
those whose parents had no high school education gained 4»7 
percentage points; blacks gained 3.6 percentage points ^ and 
those who live in the low metro communities gained 3.6 per- 
centage points. Although not as low in 1971 as other groups, 
extreme rurail ccnmunities also gcd.ned 4.1 percentage points. 

Exhibit 4 graphically shows the percentage point changes 
from 1971 to 1974. Three size-and-type of community groups 
(small plaices, medium cities and urban fringes) are not 
shown on this exhibit since their performances were so close 
to the national level. 



Exhibit 4. Changes in Basic Reading Ability 
of I7-year-Olds Prom 1971 to 1974 
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The results show that within each variable, the group 
performing least well in 1971 (region, southeast; sex, male; 
race, black; parental education, no high school; size-and- 
type of community, low metro) made substantial gains. In all 
cases, however, the 1974 mean P-values for these groups are 
still smaller than the 1971 mean P-values for the groups 
posting the next poorest performance (region, west; sex, 
female; race, white; parental education, some high school; 
size-and-type of community, extreme rural) . 



Results for Subsets of Exercises in the 1974 Mini-Assessment 

The MAPL exercises, as a set, were selected to present the 
formats of reading materials frequently encountered in everyday 
life and with which we must be able to cope if we are to be 
able to function adequately. The reading skills or behaviors 
which must be applied to the task, are for the most part, basic. 

The NAEP-MAFL staff and a panel of reading specialists 
studied the MAFL exercises and classified them into five types 
of exercise formats and five types of reading skills required. 
When a MAFL exercise contained more than one part, each part 
was classified independently. Therefore, a multipart exercise 
can represent several formats and/or several skills. 

The types of reading materials presented to 17-year-olds 
in the mini-assessment were: 

I. Passages: such as those found in stories, poems 
or newspaper and magazine articles. 

IIA. Graphic materials: drawings, pictures, signs 

and coupons. 

Ili3. Graphic materials: charts, maps and graphs. 

lie. Graphic materials: forms, such as a report card 

and a long-distance tele- 
phone bill. 

III. Reference materials: such as dictionaries, the 

telephone directory and 
encyclopedias. 

Exhibit 5 presents the mean percentages (P-values) for the 
entire set of exercises and for types of exercise formats. The 
results show that all groups performed highest on drawings, 
pictures, etc. and second highest on charts, maps and graphs. 
All groups performed third highest on passages with the exception 
of the main-big-city group, which performed fourth highest on 
this format. Most groups performed fourth highest on forms; 
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of those that did not, the west^ female r extreme-rural, medium- 
city and high-metro groups performed least well on this format 
and the main-big-city group third highest • Most groups per- 
ficzmed least well on reference materials 



Exhibit 5* Mean P^-Valties for All Exercises and 
for Types of Exercise Formats 
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87.17 86.41 8t.63 87.68 81.66- 79.09 

91.24 90.99 99.59 93.33 86.92 89.32 

9t.0O 90.97 94.7» 92.92 89.69 87.49 
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9t.90 9U 95 99.47 92.49 92.30 87.52 

92.40 93.00 99.64 94.36^ 80.99- 88.65 
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The reading^ skills assessed ware; 

1« Understanding word nwaninga.. After the person pre 
duces the sound that makes up the word^ can 
he/she understand it? 

2» Gleaning significant facts,. Can the person identify 
specific facts contained iit different kinds of 
reading materiaL? 

1» Comprehending; main ideas^ and organization^ Can. the 

reader identify the main: idee or topic: and under— 
stand how the writear organized facta to? support it2 
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4. Drawing inferences. Can the reader go beyond the 

information given by the writer and draw con- 
clusions based on that information? 

5. Reading critically. Can the reader use his own 

thoughts and experiences to analyze r criticize r 
evaluate. and then accept^ modify or reject what 
the writer has said? 

These labels attached to categories of reading tasks can 
be misleading by implying greater difficulty than the actual 
tasks display. The functional literacy reading tasks requxred 
only a basic reading skill in all categories. Only three 
exercise parts were classified in the skill category of Reading 
Critically # and therefore r summary resixlts for this small set 
of exercises are not presented here*^ 

The results for four types of reading skills presented in 
Exhibit 6 show that all groups understood word meanings best 
and drew inferences least well- While performance on the other 
behaviors was somewhat mixedr most groups performed second 
highest on canprehending main ideas and third hxghest on 
gleaming significant facts. 

Exhibit 6. Mean P-Values for All Exercises and 
for Certain Types of Reading Skills 
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General Summary of 1974 MAFL Results 

Nearly all 17 -year-old groups did moderately well on all 
questions in the mini-assessment. In only a few instances did 
less than 80 percent give a correct response, when percentages 
were adjusted to the superior reader achievement level. Even 
when the percentages were not so adjusted, they rarely dropped 
below 70 percent. All groups in the mini-assessment did best 
on reading materials involving drawings and pictures and second 
best on those concerned with charts, maps and graphs. The 
poorest performances were shown on reference materials and 
reading exercises involving forms. All groups understood word 
meanings best and drew inferences least well. Performance on 
other skills was mixed. 

In only four groups and in five reading exercise classi- 
fications did the average drop below 80 percent. 

. Blacks were below 80% on the set of all exercises 
(79.1%), forms (75.7%), reference materials (72.1%), 
gleaning significartt. facts (79.7%) and drawing 
inferences (63.95). 

. Those whose parents had no high school education were 
below 80% on reference materials (79.4%) and drawing 
inferences (74.2%). 

Those whose parents had some high school education 
were below 80% on drawing inferences (78.6%) . 

. Those in low metro areas were below 80% on reference 
materials (79.1%) and on drawing inferences (72.9%). 

Other important results for the reporting groups are con- 
trasted below. 

1. Whites did better than blacks on all of the basic 
reading exercises, with an average difference of 14.1 
percentage points in favor of whites. 

2. Girls did better than boys except in reading forms. 
The average difference between boys and girls over 
all exercises was 1.8 percentage points. 

3. Young people in the central region of the country did 
best on all kinds of basic reading tasks, and those in 
the southeast region did least well with an average 
difference of 5.2 percentage points between the two 
regions. Those in the northeast region were, second 
best on all categories except comprehension of main 
ideas, in which they ranked third; those in the west 
were third on all categories except comprehension of 
main ideas in which they ranked second. 
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4 . The yoiing people whose parents had some formal education 
past high school generally did best on all of the read- 
ing tasks, while those whose parents had no high school 
education did least well. The average difference 
between the two groups was 8.7 percentage points. Those 
who had at least one parent who graduated from high 
school ranked second, and those whose parents had some 
high school, but didn't graduate, were third. 

5. Among the sizes-and-types of commvmity, the low-metro 
group performed least well on all exercise classifi- 
cations; and the high-metro group performed highest on 
all classifications except for forms. The average 
difference between the two groups was 10.7 percentage 
points. All other groups performed very close to each 
other and the national level. 

The Mini-Assessment of Functional Literacy was designed to 
contain materials that were clearly measures of basic reading 
performance skills. The results show that for these specific 
types of materials and skills the reading achievement levels of 
17 -year-old students generally tended to be higher in 1974 than 
in 1971. 



III. THE SECOND NAEP READING ASSESSMENT 1974-1975 

Data from the second reading assessment have been collected 
and are currently being analyzed. Exercises from the first 
assessment, which have been kept secure and unpublished, were 
administered again to 9-year-olds, 13-year-olds and 17 -year-olds 
(both in-school and out-of-school) . Some of the same exercises 
which were used in the MAFL 17-year-old in-school assessment 
were included in the second Nl^P assessment. However, the 
general scope of the regular assessment is broader than the MAFL 
since it contains a wider variety "^f exercise types spread 
across a wider variety of object:. .' ./i and themes. It is not 
limited to basic reading exercise^^^ measuring functional literacy 
skills. 

Unfortiinately,- it is. riot possible to report specific results 
of the second assessment at this time. So far only a few pre- 
liminary results have been analyzed, and it is a National Assess- 
ment policy not to report results until there has been a complete 
analysis to ensure that the data cire totally accurate. 

Research has shown a close relationship between reading and 
writing abilities. National Assessment data on changes in 
writing skills has shown that, based on holistic scores for 
essays, the writing abilities of 9-year-olds have stayed about 
the same, and the abilities of 13-year-olds and 17 -year-olds 
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have decreased slightly. There was a general trend at all ages 
toward a decrease in pauragraph coherence. However, most of the 
aspects of writing generally called "mechanics" (e.g., punctuation, 
capitalization, agreements spelling, word usage and so on) eure being 
handled auiequately by the vast majority of students, and there is no 
evidence of deterioration in performance levels conceraing these 
skills. In addition, change results from the Mini-Assessment of 
Functional Literacy showed a mean nationaa increase of two percentage 
points on exercises dealing with beisic reading skills. The MAFL and 
writing- results did not reveal any evidence that schools should "go 
back to the basics"; indeed, the basics seem to be well in hand. At 
this time, although data analyses are not yet completed, NAEP sees no 
reason to be pessimistic aSscat the findings "to come regeu:«!ing changes 
in reading achievement levels for all three ages. 

In order to provide some preliminary information about resuiing 
findings, one exercise with preldLminary national percentage results 
is presented below. (The full size exercise copy presented to respon- 
dents was clear and legible.) The 17-year-old data are for in-school 
17 's. Calculations regarding the statistical significance of the 
changes are not aved.IeU3le. 
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The first: reading? change summary repoart is currentl y scheduled 
for publicatiott irt early Fall,. 19^76. Results will be presented for 
the three age levels axv±. all of the reporting groups^ A second MAFL. 
assessment wats conducted atlong with the 19-74 regular assessment* A 
report of this study will be released in: September^ 1975, 
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